Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate Versioning, Release, and PyPI Publishing #425

Open
wants to merge 4 commits into
base: next
Choose a base branch
from

Conversation

fredgrub
Copy link
Contributor

@fredgrub fredgrub commented Dec 3, 2023

Pull Request Description

This pull request introduces a Git hook and a GitHub actions workflow to automate versioning, release creation, and PyPI publishing. I have closed the previous PR, #411, leaving it for testing. If this PR passes, we will need to close the #408 PR as well. Here's a summary of the changes:

  1. GitHub Actions Workflow:

    • A workflow is triggered when a new version tag (formatted as vX.X.X) is pushed.
    • It automates the creation of a GitHub release and the publishing of a Python package to PyPI.
  2. Git Hook:

    • Added a pre-commit Git hook to extract the version from pyproject.toml and create a corresponding version tag.
  3. Poetry bumpversion plugin

Testing this PR

  • Configure ./git-hooks/ directory so that it becomes visible to git hooks.
    • git config core.hooksPath .git-hooks.
  • Install poetry-bumpversion plugin.
    • poetry self add poetry-bumpversion.
  • Update package version with poetry version <major|minor|...> (this will change both pyproject.toml and pipreqs/__init__.py).
  • Add and commit changes, including pyproject.toml and pipreqs/__init__.py.
    • This will trigger the post-commit hook.
  • Push the commit and the newly created tag.
    • For pushing a specific tag, git push <remote> <tag>.
  • Wait for the workflow termination.

Additional Notes

  • In the GitHub action secrets setting, make sure that PYPI_TOKEN exists.
  • For testing purposes, you should configure the workflow to publish to the test PyPI repository. Just add the following lines after with: on line 30.
    repository_name: "test-pypi"
    repository_url: "https://test.pypi.org/legacy/"
    
    • It's needed to create an account there. You might also want to change the token secret for TEST_PYPI_TOKEN (make sure that you create it on Test PyPI and added it on GitHub Action secrets).
    • You can also delete github releases and tags. If any was created by testing this workflow, just delete them.
  • A successefully execution of this workflow can be found at https://github.com/pipreqsxp/pipreqs/actions/runs/7064783492. The release can be found at https://test.pypi.org/project/pipreqs-build-test/.

@codecov-commenter
Copy link

codecov-commenter commented Dec 3, 2023

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.20%. Comparing base (f041de9) to head (703d809).
Report is 10 commits behind head on next.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             next     #425      +/-   ##
==========================================
+ Coverage   90.07%   90.20%   +0.12%     
==========================================
  Files           2        2              
  Lines         262      296      +34     
==========================================
+ Hits          236      267      +31     
- Misses         26       29       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

fredgrub

This comment was marked as duplicate.

fredgrub

This comment was marked as duplicate.

@fredgrub fredgrub marked this pull request as ready for review December 3, 2023 02:45
@alan-barzilay
Copy link
Collaborator

alan-barzilay commented Dec 3, 2023

Install poetry-bumpversion plugin.

poetry self add poetry-bumpversion.

isnt there a way to set this as a dev dependency? Since it is a pipy package

git config core.hooksPath .git-hooks

shouldnt we also add --local? I may be misunderstanding, but without the local flag, the hooks will be available in all git projects system wide

@fredgrub
Copy link
Contributor Author

fredgrub commented Dec 3, 2023

Install poetry-bumpversion plugin.

poetry self add poetry-bumpversion.

isn't there a way to set this as a dev dependency? Since it is a pipy package

Yeah! I searched about it and found that it is possible. I found that using self add is a way of installing the plug-in globally. If we declare the plug-in as dev dependency, it will be available only to the target project.

I'll change it ASAP.

Update 1: I was wrong. Adding a poetry plugin as a project dependency doesn't seem to be possible. See python-poetry/poetry#5729 (comment). This should be manually done by each one developing the package :(

@fredgrub
Copy link
Contributor Author

fredgrub commented Dec 8, 2023

git config core.hooksPath .git-hooks

shouldnt we also add --local? I may be misunderstanding, but without the local flag, the hooks will be available in all git projects system wide

According to git-config documentation --local is the default option (see git-config documentation).

fredgrub and others added 3 commits December 8, 2023 18:26
**TL;DR: this post-commit hook ensures that valid versions are automatically tagged based on changes in the `pyproject.toml` file.**

This post-commit hook automates the tagging of the project based on changes in the `pyproject.toml` file. Here's a breakdown of what it does:

**1. Extracting the version number:**

* `git diff HEAD^..HEAD`: This line compares the current commit (`HEAD`) to its immediate predecessor (`HEAD^`).
* `-- "$(git rev-parse --show-toplevel)"/pyproject.toml`: This specifies the `pyproject.toml` file within the project root directory.
* `grep -m 1 '^\+.*version'`: This searches for the first line starting with a "+" and containing the word "version".
* `sed -s 's/[^A-Z0-9\.\-]//g'`: This removes any characters except numbers, letters, dots, and hyphens from the matched line.
* `version=`: Finally, the extracted version string is stored in the `version` variable.

**2. Validating the version format:**

* `if [[ ! $version =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)(\-[A-Z]+\.[0-9]+)?$ ]]; then`: This checks if the extracted version string matches a specific format:
    * `^`: Starts with the beginning of the string.
    * `([0-9]+)`: Matches one or more digits (major version).
    * `\.`: Matches a literal dot.
    * `([0-9]+)`: Matches one or more digits (minor version).
    * `\.`: Matches a literal dot.
    * `([0-9]+)`: Matches one or more digits (patch version).
    * `(\-[A-Z]+\.[0-9]+)?`: This is optional and matches a hyphen followed by one or more uppercase letters and a dot and another number (pre-release + build information).
    * `$`: Matches the end of the string.
* If the format is invalid, it logs a message and exits with an error code.

**3. Creating the tag:**

* `git tag -a "v$version"`: This creates a new annotated tag named `v$version` (with the prefix "v") using the extracted version number.
* ``-m "`git log -1 --format=%s`"``: This sets the tag message with the full commit message of the current commit.
* `echo "Created a new tag, v$version"`: This prints a confirmation message.

Co-authored-by: Darwish Ahmad Herati <[email protected]>
This GitHub Actions workflow automates two tasks:

1. **Create Release:**
   - Triggered when a new version tag (formatted as `vX.X.X`) is pushed.
   - Creates a release with the tag name, marking it as the latest version.

2. **Publish to PyPI:**
   - Runs after the release is created successfully.
   - Builds and publishes the Python package to PyPI using Poetry.
   - Ignores development requirements during the publishing process.

This ensures that each new version gets a release on GitHub and the corresponding Python package is published on PyPI.

Co-authored-by: Darwish Ahmad Herati <[email protected]>
Co-authored-by: mateuslatrova <[email protected]>
- Add instructions to setup git-hooks and poetry-bumpversion plugin
Copy link
Collaborator

@alan-barzilay alan-barzilay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

everything looks good to me! I will not merge it yet due to the secret not being set yet

@alan-barzilay
Copy link
Collaborator

Since I have no permission to set the secret, I have been reaching out to @bndr to set it but I've got no response for the last couple months. Maybe he is busy, but this wont be merged until the secret is set so sadly this pr might be dead. I truly appreciate all the work put into it and apologize for this, I probably should have explored this issue before allowing work to be done on this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants